Limbs detection and tracking of head-fixed mice for behavioral phenotyping using motion tubes and deep learning

The broad accessibility of affordable and reliable recording equipment and its relative ease of use has enabled neuroscientists to record large amounts of neurophysiological and behavioral data. Given that most of this raw data is unlabeled, great effort is required to adapt it for behavioral phenotyping or signal extraction, for behavioral and neurophysiological data, respectively. Traditional methods for labeling datasets rely on human annotators which is a resource and time intensive process, which often produce data that that is prone to reproducibility errors. Here, we propose a deep learning-based image segmentation framework to automatically extract and label limb movements from movies capturing frontal and lateral views of head-fixed mice. The method decomposes the image into elemental regions (superpixels) with similar appearance and concordant dynamics and stacks them following their partial temporal trajectory. These 3D descriptors (referred as motion cues) are used to train a deep convolutional neural network (CNN). We use the features extracted at the last fully connected layer of the network for training a Long Short Term Memory (LSTM) network that introduces spatio-temporal coherence to the limb segmentation. We tested the pipeline in two video acquisition settings. In the first, the camera is installed on the right side of the mouse (lateral setting). In the second, the camera is installed facing the mouse directly (frontal setting). We also investigated the effect of the noise present in the videos and the amount of training data needed, and we found that reducing the number of training samples does not result in a drop of more than 5% in detection accuracy even when as little as 10% of the available data is used for training

Enabling automatic just-in-time evaluation of in-class discussions in on-line collaborative learning practices

Learning by discussion when applied to on-line collaborative learning settings can provide significant benefits for students in education in general. Indeed, the discussion process plays an important social task in collaborative learning practices. Participants can discuss about the activity being performed, collaborate with each other through the exchange of ideas that may arise, propose new resolution mechanisms, justify and refine their own contributions, and as a result, acquire new knowledge. Considering these benefits, current educational organizations incorporate on-line discussions into web-based courses as part of the very rationale of their pedagogical models. However, in-class collaborative assignments are usually greatly participated and contributed, which makes the monitoring and assessment tasks by tutors and moderators time-consuming, tedious and error-prone. Specially hard if not impossible by human tutors is to manually deal with the sequences of hundreds of contributions making up the discussion threads and the relations between these contributions. Consequently, tutoring tasks during on-line discussions usually restrict to offer evaluation results of the contributing effort and quality after the collaborative learning activity takes place and thus neglect the essential issue of constantly considering the process of knowledge building while it is still being performed. In this paper, we propose a multidimensional model based on data analysis from online collaborative discussion interactions that provides a first step towards an automatic evaluation in just-in-time fashion. The context of this study is a real on-line discussion experience that took place at the Open University of Catalonia.Peer ReviewedPostprint (published version

Combining statistical learning with metaheuristics for the multi-depot vehicle routing problem with market segmentation

In real-life logistics and distribution activities it is usual to face situations in which the distribution of goods has to be made from multiple warehouses or depots to the nal customers. This problem is known as the Multi-Depot Vehicle Routing Problem (MDVRP), and it typically includes two sequential and correlated stages: (a) the assignment map of customers to depots, and (b) the corresponding design of the distribution routes. Most of the existing work in the literature has focused on minimizing distance-based distribution costs while satisfying a number of capacity constraints. However, no attention has been given so far to potential variations in demands due to the tness of the customerdepot mapping in the case of heterogeneous depots. In this paper, we consider this realistic version of the problem in which the depots are heterogeneous in terms of their commercial o er and customers show di erent willingness to consume depending on how well the assigned depot ts their preferences. Thus, we assume that di erent customer-depot assignment maps will lead to di erent customer-expenditure levels. As a consequence, market-segmentation strategies need to be considered in order to increase sales and total income while accounting for the distribution costs. To solve this extension of the MDVRP, we propose a hybrid approach that combines statistical learning techniques with a metaheuristic framework. First, a set of predictive models is generated from historical data. These statistical models allow estimating the demand of any customer depending on the assigned depot. Then, the estimated expenditure of each customer is included as part of an enriched objective function as a way to better guide the stochastic local search inside the metaheuristic framework. A set of computational experiments contribute to illustrate our approach and how the extended MDVRP considered here diré in terms of the proposed solutions from the traditional one.Peer ReviewedPreprin

Deep Learning of Retinal Imaging: A Useful Tool for Coronary Artery Calcium Score Prediction in Diabetic Patients

Retina fundus imaging; Deep learning; Medical imagingImatge del fons de la retina; Aprenentatge profund; Imatges mèdiquesImagen del fondo de la retina; Aprendizaje profundo; Imágenes médicasCardiovascular diseases (CVD) are one of the leading causes of death in the developed countries. Previous studies suggest that retina blood vessels provide relevant information on cardiovascular risk. Retina fundus imaging (RFI) is a cheap medical imaging test that is already regularly performed in diabetic population as screening of diabetic retinopathy (DR). Since diabetes is a major cause of CVD, we wanted to explore the use Deep Learning architectures on RFI as a tool for predicting CV risk in this population. Particularly, we use the coronary artery calcium (CAC) score as a marker, and train a convolutional neural network (CNN) to predict whether it surpasses a certain threshold defined by experts. The preliminary experiments on a reduced set of clinically verified patients show promising accuracies. In addition, we observed that elementary clinical data is positively correlated with the risk of suffering from a CV disease. We found that the results from both informational cues are complementary, and we propose two applications that can benefit from the combination of image analysis and clinical data.This research was funded by “RTI2018-095232-B-C22” grant from the Spanish Ministry of Science, Innovation and Universities (FEDER funds)

Face classification using discriminative features and classifier combination

Consultable des del TDXTítol obtingut de la portada digitalitzadaA mesura que la tecnologia evoluciona, apareixen noves aplicacions en el mon de la classificació facial. En el reconeixement de patrons, normalment veiem les cares com a punts en un espai de alta dimensionalitat definit pels valors dels seus pixels. Aquesta aproximació pateix diversos problemes: el fenomen de la «la maledicció de la dimensionalitat», la presència d'oclusions parcials o canvis locals en la il·luminació. Tradicionalment, només les característiques internes de les imatges facials s'han utilitzat per a classificar, on normalment es fa una extracció de característiques. Les tècniques d'extracció de característiques permeten reduir la influencia dels problemes mencionats, reduint també el soroll inherent de les imatges naturals alhora que es poden aprendre característiques invariants de les imatges facials. En la primera part d'aquesta tesi presentem alguns mètodes d'extracció de característiques clàssics: Anàlisi de Components Principals (PCA), Anàlisi de Components Independents (ICA), Factorització No Negativa de Matrius (NMF), i l'Anàlisi Discriminant de Fisher (FLD), totes elles fent alguna mena d'assumpció en les dades a classificar. La principal contribució d'aquest treball es una nova família de tècniques d'extracció de característiques usant el algorisme del Adaboost. El nostre mètode no fa cap assumpció en les dades a classificar, i construeix de forma incremental la matriu de projecció tenint en compte els exemples mes difícils Per altra banda, en la segon apart de la tesi explorem el rol de les característiques externes en el procés de classificació facial, i presentem un nou mètode per extreure un conjunt alineat de característiques a partir de la informació externa que poden ser combinades amb les tècniques clàssiques millorant els resultats globals de classificació.As technology evolves, new applications dealing with face classification appear. In pattern recognition, faces are usually seen as points in a high dimensional spaces defined by their pixel values. This approach must deal with several problems such as: the curse of dimensionality, the presence of partial occlusions or local changes in the illumination. Traditionally, only the internal features of face images have been used for classification purposes, where usually a feature extraction step is performed. Feature extraction techniques allow to reduce the influence of the problems mentioned, reducing also the noise inherent from natural images and learning invariant characteristics from face images. In the first part of this thesis some internal feature extraction methods are presented: Principal Component Analysis (PCA), Independent Component Analysis (ICA), Non Negative Matrix Factorization (NMF), and Fisher Linear Discriminant Analysis (FLD), all of them making some kind of the assumption on the data to classify. The main contribution of our work is a non parametric feature extraction family of techniques using the Adaboost algorithm. Our method makes no assumptions on the data to classify, and incrementally builds the projection matrix taking into account the most difficult samples. On the other hand, in the second part of this thesis we also explore the role of external features in face classification purposes, and present a method for extracting an aligned feature set from external face information that can be combined with the classic internal features improving the global performance of the face classification task

Hand detection in cluttered scene images using Fourier-Mellin invariant features

This paper proposes an automatic hand detection system that combines the Fourier-Mellin Transform along with other computer vision techniques to achieve hand detection in cluttered scene color images. The proposed system uses the Fourier-Mellin Transform as an invariant feature extractor to perform RST invariant hand detection. In a first stage of the system a simple non-adaptive skin color-based image segmentation and an interest point detector based on corners are used in order to identify regions of interest that contains possible matches. A sliding window algorithm is then used to scan the image at different scales performing the FMT calculations only in the previously detected regions of interest and comparing the extracted FM descriptor of the windows with a hand descriptors database obtained from a train image set. The results of the performed experiments suggest the use of Fourier-Mellin invariant features as a promising approach for automatic hand detection

A Hierarchical Approach for Multi-task Logistic Regression

Peer-reviewedIn the statistical pattern recognition eld the number of samples to train a classifer is usually insu cient. Nevertheless, it has been shown that some learning domains can be divided in a set of related tasks, that can be simultaneously trained sharing information among the different tasks. This methodology is known as the multi-task learning paradigm. In this paper we propose a multi-task probabilistic logistic regression model and develop a learning algorithm based in this framework, which can deal with the small sample size problem. Our experiments performed in two independent databases from the UCI and a multi-task face classification experiment show the improved accuracies of the multi-task learning approach with respect to the single task approach when using the same probabilistic model

Shared Regularization Techniques for Logistic Regression. Computer Vision : advances in research and development

Shared Regularization Techniques for Logistic Regression. Computer Vision : advances in research and developmen

Multitask learning : An application to incremental face recognition

Usually face classification applications suffer from two important problems: the number of training samples from each class is reduced, and the final system usually must be extended to incorporate new people to recognize. In this paper we introduce a face recognition method that extends a previous boosting-based classifier adding new classes and avoiding the need of retraining the system each time a new person joins the system.The classifier is trained using the multitask learning principle and multiple verification tasks are trained together sharing the same feature space. The new classes are added taking advantage of the previous learned structure, being the addition of new classes not computationally demanding. Our experiments with two differ- ent data sets show that the performance does not decrease drastically even when the number of classes of the base problem is multiplied by a factor of 8

Measuring external face appearance for face classification

Face recognition and subject verification. In a first approach, we show that the external features encoded in the NMF coefficients yield enough useful information for classification purposes. Then we compare the information contributed by the external features and the internal features. Finally, the last step is to combine the information provided by the external and the internal features. We show that both kinds of information are complementary, providing and extra information cue that can improve the classification results in presence of occlusions and local changes in the illumination